Profiling Report Diamonds Data

Dataset statistics

Number of variables	10
Number of observations	53930
Missing cells	0
Missing cells (%)	0.0%
Duplicate rows	146
Duplicate rows (%)	0.3%
Total size in memory	3.9 MiB
Average record size in memory	76.0 B

Variable types

NUM	10

Warnings

Dataset has 146 (0.3%) duplicate rows	Duplicates
`price` is highly correlated with `carat`	High correlation
`carat` is highly correlated with `price` and 3 other fields	High correlation
`x` is highly correlated with `carat` and 2 other fields	High correlation
`y` is highly correlated with `carat` and 2 other fields	High correlation
`z` is highly correlated with `carat` and 2 other fields	High correlation

Reproduction

Analysis started	2020-09-06 22:33:21.422037
Analysis finished	2020-09-06 22:33:48.434916
Duration	27.01 seconds
Software version	pandas-profiling v2.9.0
Download configuration	config.yaml

carat
Real number (ℝ_≥0)

HIGH CORRELATION

Distinct	273
Distinct (%)	0.5%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%

Mean	0.7979757093
Minimum	0.2
Maximum	5.01
Zeros	0
Zeros (%)	0.0%
Memory size	421.3 KiB

Quantile statistics

Minimum	0.2
5-th percentile	0.3
Q1	0.4
median	0.7
Q3	1.04
95-th percentile	1.7
Maximum	5.01
Range	4.81
Interquartile range (IQR)	0.64

Descriptive statistics

Standard deviation	0.4740348567
Coefficient of variation (CV)	0.594046725
Kurtosis	1.25617613
Mean	0.7979757093
Median Absolute Deviation (MAD)	0.32
Skewness	1.116517792
Sum	43034.83
Variance	0.2247090454
Monotocity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
0.3	2604	4.8%
0.31	2249	4.2%
1.01	2242	4.2%
0.7	1981	3.7%
0.32	1839	3.4%
1	1558	2.9%
0.9	1484	2.8%
0.41	1382	2.6%
0.4	1298	2.4%
0.71	1293	2.4%
Other values (263)	36000	66.8%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
0.2	12	< 0.1%
0.21	9	< 0.1%
0.22	5	< 0.1%
0.23	293	0.5%
0.24	254	0.5%

Value	Count	Frequency (%)
5.01	1	< 0.1%
4.5	1	< 0.1%
4.13	1	< 0.1%
4.01	2	< 0.1%
4	1	< 0.1%

cut
Real number (ℝ_≥0)

Distinct	5
Distinct (%)	< 0.1%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%

Mean	3.90406082
Minimum	1
Maximum	5
Zeros	0
Zeros (%)	0.0%
Memory size	421.3 KiB

Quantile statistics

Minimum	1
5-th percentile	2
Q1	3
median	4
Q3	5
95-th percentile	5
Maximum	5
Range	4
Interquartile range (IQR)	2

Descriptive statistics

Standard deviation	1.116593133
Coefficient of variation (CV)	0.2860081295
Kurtosis	-0.3979206693
Mean	3.90406082
Median Absolute Deviation (MAD)	1
Skewness	-0.7171326869
Sum	210546
Variance	1.246780224
Monotocity	Not monotonic

Histogram with fixed size bins (bins=5)

Value	Count	Frequency (%)
5	21546	40.0%
4	13788	25.6%
3	12082	22.4%
2	4904	9.1%
1	1610	3.0%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
1	1610	3.0%
2	4904	9.1%
3	12082	22.4%
4	13788	25.6%
5	21546	40.0%

Value	Count	Frequency (%)
5	21546	40.0%
4	13788	25.6%
3	12082	22.4%
2	4904	9.1%
1	1610	3.0%

color
Real number (ℝ_≥0)

Distinct	7
Distinct (%)	< 0.1%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%

Mean	3.594103467
Minimum	1
Maximum	7
Zeros	0
Zeros (%)	0.0%
Memory size	421.3 KiB

Quantile statistics

Minimum	1
5-th percentile	1
Q1	2
median	4
Q3	5
95-th percentile	7
Maximum	7
Range	6
Interquartile range (IQR)	3

Descriptive statistics

Standard deviation	1.701080135
Coefficient of variation (CV)	0.4732974858
Kurtosis	-0.8667626966
Mean	3.594103467
Median Absolute Deviation (MAD)	1
Skewness	0.1894017457
Sum	193830
Variance	2.893673625
Monotocity	Not monotonic

Histogram with fixed size bins (bins=7)

Value	Count	Frequency (%)
4	11288	20.9%
2	9796	18.2%
3	9541	17.7%
5	8304	15.4%
1	6774	12.6%
6	5420	10.1%
7	2807	5.2%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
1	6774	12.6%
2	9796	18.2%
3	9541	17.7%
4	11288	20.9%
5	8304	15.4%

Value	Count	Frequency (%)
7	2807	5.2%
6	5420	10.1%
5	8304	15.4%
4	11288	20.9%
3	9541	17.7%

clarity
Real number (ℝ_≥0)

Distinct	8
Distinct (%)	< 0.1%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%

Mean	5.949119229
Minimum	2
Maximum	9
Zeros	0
Zeros (%)	0.0%
Memory size	421.3 KiB

Quantile statistics

Minimum	2
5-th percentile	3
Q1	5
median	6
Q3	7
95-th percentile	8
Maximum	9
Range	7
Interquartile range (IQR)	2

Descriptive statistics

Standard deviation	1.647180648
Coefficient of variation (CV)	0.2768780696
Kurtosis	-0.3945409522
Mean	5.949119229
Median Absolute Deviation (MAD)	1
Skewness	-0.5516774328
Sum	320836
Variance	2.713204086
Monotocity	Not monotonic

Histogram with fixed size bins (bins=8)

Value	Count	Frequency (%)
7	13065	24.2%
6	12256	22.7%
8	9193	17.0%
5	8167	15.1%
4	5063	9.4%
3	3655	6.8%
2	1790	3.3%
9	741	1.4%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
2	1790	3.3%
3	3655	6.8%
4	5063	9.4%
5	8167	15.1%
6	12256	22.7%

Value	Count	Frequency (%)
9	741	1.4%
8	9193	17.0%
7	13065	24.2%
6	12256	22.7%
5	8167	15.1%

depth
Real number (ℝ_≥0)

Distinct	184
Distinct (%)	0.3%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%

Mean	61.74932505
Minimum	43
Maximum	79
Zeros	0
Zeros (%)	0.0%
Memory size	421.3 KiB

Quantile statistics

Minimum	43
5-th percentile	59.3
Q1	61
median	61.8
Q3	62.5
95-th percentile	63.8
Maximum	79
Range	36
Interquartile range (IQR)	1.5

Descriptive statistics

Standard deviation	1.432710583
Coefficient of variation (CV)	0.02320204443
Kurtosis	5.738791716
Mean	61.74932505
Median Absolute Deviation (MAD)	0.7
Skewness	-0.08216973706
Sum	3330141.1
Variance	2.052659616
Monotocity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
62	2239	4.2%
61.9	2163	4.0%
61.8	2077	3.9%
62.2	2039	3.8%
62.1	2017	3.7%
61.6	1955	3.6%
62.3	1940	3.6%
61.7	1904	3.5%
62.4	1791	3.3%
61.5	1718	3.2%
Other values (174)	34087	63.2%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
43	2	< 0.1%
44	1	< 0.1%
50.8	1	< 0.1%
51	1	< 0.1%
52.2	1	< 0.1%

Value	Count	Frequency (%)
79	2	< 0.1%
78.2	1	< 0.1%
73.6	1	< 0.1%
72.9	1	< 0.1%
72.2	1	< 0.1%

table
Real number (ℝ_≥0)

Distinct	127
Distinct (%)	0.2%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%

Mean	57.45732802
Minimum	43
Maximum	95
Zeros	0
Zeros (%)	0.0%
Memory size	421.3 KiB

Quantile statistics

Minimum	43
5-th percentile	54
Q1	56
median	57
Q3	59
95-th percentile	61
Maximum	95
Range	52
Interquartile range (IQR)	3

Descriptive statistics

Standard deviation	2.234577995
Coefficient of variation (CV)	0.03889108791
Kurtosis	2.80171247
Mean	57.45732802
Median Absolute Deviation (MAD)	1
Skewness	0.7968352098
Sum	3098673.7
Variance	4.993338815
Monotocity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
56	9879	18.3%
57	9722	18.0%
58	8368	15.5%
59	6570	12.2%
55	6266	11.6%
60	4241	7.9%
54	2594	4.8%
61	2282	4.2%
62	1273	2.4%
63	588	1.1%
Other values (117)	2147	4.0%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
43	1	< 0.1%
44	1	< 0.1%
49	2	< 0.1%
50	2	< 0.1%
50.1	1	< 0.1%

Value	Count	Frequency (%)
95	1	< 0.1%
79	1	< 0.1%
76	1	< 0.1%
73	4	< 0.1%
71	1	< 0.1%

price
Real number (ℝ_≥0)

HIGH CORRELATION

Distinct	11602
Distinct (%)	21.5%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%

Mean	3933.054942
Minimum	326
Maximum	18823
Zeros	0
Zeros (%)	0.0%
Memory size	210.7 KiB

Quantile statistics

Minimum	326
5-th percentile	544
Q1	950
median	2401
Q3	5325
95-th percentile	13108.1
Maximum	18823
Range	18497
Interquartile range (IQR)	4375

Descriptive statistics

Standard deviation	3989.628569
Coefficient of variation (CV)	1.014384144
Kurtosis	2.177188035
Mean	3933.054942
Median Absolute Deviation (MAD)	1670
Skewness	1.618288043
Sum	212109653
Variance	15917136.12
Monotocity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
605	132	0.2%
802	127	0.2%
625	126	0.2%
828	124	0.2%
776	124	0.2%
698	121	0.2%
789	121	0.2%
544	120	0.2%
666	114	0.2%
552	113	0.2%
Other values (11592)	52708	97.7%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
326	2	< 0.1%
327	1	< 0.1%
334	1	< 0.1%
335	1	< 0.1%
336	2	< 0.1%

Value	Count	Frequency (%)
18823	1	< 0.1%
18818	1	< 0.1%
18806	1	< 0.1%
18804	1	< 0.1%
18803	1	< 0.1%

x
Real number (ℝ_≥0)

HIGH CORRELATION

Distinct	554
Distinct (%)	1.0%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%

Mean	5.731236232
Minimum	0
Maximum	10.74
Zeros	8
Zeros (%)	< 0.1%
Memory size	421.3 KiB

Quantile statistics

Minimum	0
5-th percentile	4.29
Q1	4.71
median	5.7
Q3	6.54
95-th percentile	7.66
Maximum	10.74
Range	10.74
Interquartile range (IQR)	1.83

Descriptive statistics

Standard deviation	1.121807157
Coefficient of variation (CV)	0.195735634
Kurtosis	-0.6183150452
Mean	5.731236232
Median Absolute Deviation (MAD)	0.93
Skewness	0.3785630303
Sum	309085.57
Variance	1.258451299
Monotocity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
4.37	448	0.8%
4.34	437	0.8%
4.33	429	0.8%
4.38	428	0.8%
4.32	425	0.8%
4.35	407	0.8%
4.39	388	0.7%
4.31	387	0.7%
4.36	386	0.7%
4.4	373	0.7%
Other values (544)	49822	92.4%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
0	8	< 0.1%
3.73	2	< 0.1%
3.74	1	< 0.1%
3.76	1	< 0.1%
3.77	1	< 0.1%

Value	Count	Frequency (%)
10.74	1	< 0.1%
10.23	1	< 0.1%
10.14	1	< 0.1%
10.02	1	< 0.1%
10.01	1	< 0.1%

y
Real number (ℝ_≥0)

HIGH CORRELATION

Distinct	552
Distinct (%)	1.0%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%

Mean	5.734600593
Minimum	0
Maximum	58.9
Zeros	7
Zeros (%)	< 0.1%
Memory size	421.3 KiB

Quantile statistics

Minimum	0
5-th percentile	4.3
Q1	4.72
median	5.71
Q3	6.54
95-th percentile	7.65
Maximum	58.9
Range	58.9
Interquartile range (IQR)	1.82

Descriptive statistics

Standard deviation	1.142184369
Coefficient of variation (CV)	0.1991741797
Kurtosis	91.21490696
Mean	5.734600593
Median Absolute Deviation (MAD)	0.92
Skewness	2.434172149
Sum	309267.01
Variance	1.304585133
Monotocity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
4.34	437	0.8%
4.37	435	0.8%
4.35	425	0.8%
4.33	421	0.8%
4.32	414	0.8%
4.39	407	0.8%
4.38	406	0.8%
4.31	386	0.7%
4.4	386	0.7%
4.41	384	0.7%
Other values (542)	49829	92.4%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
0	7	< 0.1%
3.68	1	< 0.1%
3.71	2	< 0.1%
3.72	1	< 0.1%
3.73	1	< 0.1%

Value	Count	Frequency (%)
58.9	1	< 0.1%
31.8	1	< 0.1%
10.54	1	< 0.1%
10.16	1	< 0.1%
10.1	1	< 0.1%

z
Real number (ℝ_≥0)

HIGH CORRELATION

Distinct	375
Distinct (%)	0.7%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%

Mean	3.538776377
Minimum	0
Maximum	31.8
Zeros	20
Zeros (%)	< 0.1%
Memory size	421.3 KiB

Quantile statistics

Minimum	0
5-th percentile	2.65
Q1	2.91
median	3.53
Q3	4.04
95-th percentile	4.73
Maximum	31.8
Range	31.8
Interquartile range (IQR)	1.13

Descriptive statistics

Standard deviation	0.705729344
Coefficient of variation (CV)	0.1994275051
Kurtosis	47.08679971
Mean	3.538776377
Median Absolute Deviation (MAD)	0.57
Skewness	1.522387342
Sum	190846.21
Variance	0.498053907
Monotocity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
2.7	767	1.4%
2.69	748	1.4%
2.71	738	1.4%
2.68	730	1.4%
2.72	697	1.3%
2.67	649	1.2%
2.73	612	1.1%
2.66	555	1.0%
2.74	547	1.0%
4.02	538	1.0%
Other values (365)	47349	87.8%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
0	20	< 0.1%
1.07	1	< 0.1%
1.41	1	< 0.1%
1.53	1	< 0.1%
2.06	1	< 0.1%

Value	Count	Frequency (%)
31.8	1	< 0.1%
8.06	1	< 0.1%
6.98	1	< 0.1%
6.72	1	< 0.1%
6.43	1	< 0.1%

Pearson's r

The Pearson's correlation coefficient (r) is a measure of linear correlation between two variables. It's value lies between -1 and +1, -1 indicating total negative linear correlation, 0 indicating no linear correlation and 1 indicating total positive linear correlation. Furthermore, r is invariant under separate changes in location and scale of the two variables, implying that for a linear function the angle to the x-axis does not affect r.

To calculate r for two variables X and Y, one divides the covariance of X and Y by the product of their standard deviations.

Spearman's ρ

The Spearman's rank correlation coefficient (ρ) is a measure of monotonic correlation between two variables, and is therefore better in catching nonlinear monotonic correlations than Pearson's r. It's value lies between -1 and +1, -1 indicating total negative monotonic correlation, 0 indicating no monotonic correlation and 1 indicating total positive monotonic correlation.

To calculate ρ for two variables X and Y, one divides the covariance of the rank variables of X and Y by the product of their standard deviations.

Kendall's τ

Similarly to Spearman's rank correlation coefficient, the Kendall rank correlation coefficient (τ) measures ordinal association between two variables. It's value lies between -1 and +1, -1 indicating total negative correlation, 0 indicating no correlation and 1 indicating total positive correlation.

To calculate τ for two variables X and Y, one determines the number of concordant and discordant pairs of observations. τ is given by the number of concordant pairs minus the discordant pairs divided by the total number of pairs.

Phik (φk)

Phik (φk) is a new and practical correlation coefficient that works consistently between categorical, ordinal and interval variables, captures non-linear dependency and reverts to the Pearson correlation coefficient in case of a bivariate normal input distribution. There is extensive documentation available here.

Count
Matrix

First rows

	carat	cut	color	clarity	depth	table	price	x	y	z
0	0.23	5	2	8	61.5	55.0	326	3.95	3.98	2.43
1	0.21	4	2	7	59.8	61.0	326	3.89	3.84	2.31
2	0.23	2	2	5	56.9	65.0	327	4.05	4.07	2.31
3	0.29	4	6	6	62.4	58.0	334	4.20	4.23	2.63
4	0.31	2	7	8	63.3	58.0	335	4.34	4.35	2.75
5	0.24	3	7	4	62.8	57.0	336	3.94	3.96	2.48
6	0.24	3	6	3	62.3	57.0	336	3.95	3.98	2.47
7	0.26	3	5	7	61.9	55.0	337	4.07	4.11	2.53
8	0.22	1	2	6	65.1	61.0	337	3.87	3.78	2.49
9	0.23	3	5	5	59.4	61.0	338	4.00	4.05	2.39

Last rows

	carat	cut	color	clarity	depth	table	price	x	y	z
53920	0.71	4	2	7	60.5	55.0	2756	5.79	5.74	3.49
53921	0.71	4	3	7	59.8	62.0	2756	5.74	5.73	3.43
53922	0.70	3	2	6	60.5	59.0	2757	5.71	5.76	3.47
53923	0.70	3	2	6	61.2	59.0	2757	5.69	5.72	3.49
53924	0.72	4	1	7	62.7	59.0	2757	5.69	5.73	3.58
53925	0.72	5	1	7	60.8	57.0	2757	5.75	5.76	3.50
53926	0.72	2	1	7	63.1	55.0	2757	5.69	5.75	3.61
53927	0.70	3	1	7	62.8	60.0	2757	5.66	5.68	3.56
53928	0.86	4	5	8	61.0	58.0	2757	6.15	6.12	3.74
53929	0.75	5	1	8	62.2	55.0	2757	5.83	5.87	3.64

Most frequent

	carat	cut	color	clarity	depth	table	price	x	y	z	count
84	0.79	5	4	7	62.3	57.0	2898	5.90	5.85	3.66	5
0	0.30	2	7	5	63.4	57.0	394	4.23	4.26	2.69	2
1	0.30	3	4	6	63.0	55.0	526	4.29	4.31	2.71	2
2	0.30	3	7	5	63.4	57.0	506	4.26	4.23	2.69	2
3	0.30	4	1	7	62.2	58.0	709	4.31	4.28	2.67	2
4	0.30	5	4	2	62.1	55.0	863	4.32	4.35	2.69	2
5	0.30	5	4	6	63.0	55.0	675	4.31	4.29	2.71	2
6	0.30	5	5	7	62.2	57.0	450	4.26	4.29	2.66	2
7	0.30	5	5	7	62.2	57.0	450	4.27	4.28	2.66	2
8	0.31	2	1	7	63.5	56.0	571	4.29	4.31	2.73	2

Overview

Variables

Interactions

Correlations

Pearson's r

Spearman's ρ

Kendall's τ

Phik (φk)

Missing values

Sample

First rows

Last rows

Duplicate rows

Most frequent